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Abstract 

United  States  Department  of  Defense  (DoD)  space  acquisition  programs  almost 
always  experience  significant  schedule  growth,  which  closely  correlates  to  cost  growth. 
Even  with  the  advent  of  more  reliable  launch  vehicles,  schedule  delays  often  exceed  3 
years  and  have  the  implication  of  reduced  military  or  national  security  capabilities, 
significant  increases  in  costs  and  occasionally  program  cancellations.  This  paper  is 
intended  to  give  acquisition  professionals  insight  into  the  DoD’s  space  launch  process 
through  modeling  and  simulation.  It  discusses  the  reasons  a  model  is  needed,  outlining 
the  perceived  causes  and  resulting  impacts  of  significant  schedule  growth  from  baseline 
planned  launch  dates  to  actual  launch  dates  for  satellites.  This  paper  scopes  the  problem 
into  a  practical  area  of  research,  specifically  from  acquisition  Milestone  C  through 
launch.  Significant  drivers  to  space  vehicle  timelines  are  the  processes  associated  with 
scheduling  launch  support  and  conducting  integration  efforts  for  launch  processing. 
Seven  causal  factors  are  identified  as  areas  of  delay.  These  factors  are  analyzed  and 
assessed  to  draw  conclusions  about  schedule  growth  and  timeline  considerations.  The 
authors  discuss  the  implications  of  these  factors  and  hypothesize  about  lower-level 
contributors  to  create  recommendations  for  those  involved  with  space  policy, 
acquisitions,  and  launch.  Lastly,  recommendations  are  provided  to  focus  future  research 
towards  the  identification  of  specific  actions,  which  may  be  taken  to  reduce  schedule 
delay  occurring  between  Milestone  C  and  the  launch  of  a  space  system. 
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I.  Introduction 


1.1  Problem 

United  States  Department  of  Defense  (DoD)  space  acquisition  programs  are 
almost  always  plagued  with  significant  unplanned  schedule  growth.  Cost  and  schedule 
for  required  launch  capabilities  are  determined  early  in  the  acquisition  process  based  on 
requirements.  An  increase  in  space  launch  schedule  affects  not  only  the  individual 
program,  but  often  requires  funding  transfers  from  other  space  acquisition  programs. 

This  has  the  potential  to  affect  all  of  the  Department’s  space  acquisition  programs  in 
tenns  of  budget  and  schedule.  The  impact  may  even  lead  to  the  termination  of  one  or 
more  programs  altogether. 

The  Government  Accountability  Office  (GAO)  conducted  a  study  of  multiple 
space  programs  in  2006  to  detennine  the  cause  of  increased  costs  to  programs.  Not 
surprisingly,  the  study  revealed  that,  “...original  cost  [and  schedule]  estimates  were 
particularly  unrealistic  about  the  promise  of  savings  from  increased  contractor  program 
management  responsibilities,  the  constancy  and  availability  of  the  industrial  base,  savings 
that  could  be  accrued  from  heritage  systems,  the  amount  of  weight  growth  that  would 
occur  during  a  program,  the  availability  of  mature  technology,  the  stability  of  funding, 
the  stability  of  requirements,  and  the  achievability  of  planned  schedules ”  (Government 
Accountability  Office,  2006).  Moreover,  these  factors  are  intricately  tied  to  one  another 
when  assessed  from  the  vantage  point  of  a  System  Program  Office  (SPO).  For  example, 
the  unavailability  of  mature  technology  leads  to  longer  schedules  while  the  necessary 
technology  is  developed. 
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An  inconsistent  industrial  base  may  also  cause  parts  availability  issues,  leading  to 
longer  acceptance  times  for  simple  components  that  now  must  be  re-qualified  for  space 
flight.  Even  the  reuse  of  heritage  designs  often  requires  rigorous  testing  at  critical 
milestones  with  new  technologies  and  designs  to  ensure  performance  and  safety  margins 
are  still  maintained  for  the  system. 

Since  the  1970’s  the  acquisition  cost  for  space  launch  has  increased  dramatically. 
Many  economic  studies  were  conducted  to  better  predict  the  cost  for  space  launch  based 
on  historical  data.  While  these  studies  helped  predict  actual  costs  more  precisely  they  did 
nothing  to  prevent  the  overall  acquisition  cost  for  space  launch  from  rising  (Hertzfeld, 
2005). 

In  the  early  1990s,  the  United  States  suffered  a  series  of  critical  setbacks  with 
respect  to  national  security.  The  cause  of  these  setbacks  was  the  failure  of  several 
satellite  launch  systems  that  were  intended  to  support  DoD  and  National  Intelligence 
Community  (IC)  missions.  In  total,  three  Titan  IV  mishaps  resulted  in  nearly  $3  billion  in 
fiscal  losses,  with  two  Air  Force  and  one  National  Reconnaissance  Office  (NRO) 
satellites  being  lost.  Additionally,  the  Delta  III  launch  vehicle  experienced  two  failures 
during  commercial  launches.  Besides  the  launch  failures,  several  in-flight  anomalies 
occurred,  casting  a  shadow  of  doubt  on  the  Nation’s  ability  to  guarantee  assured  access  to 
space  (US  Air  Force,  1999). 

As  a  result  of  unacceptable  failure  rates,  the  DoD  introduced  major  changes  in  the 
risk  posture  associated  with  space  launch.  In  August  2008,  then  Maj  Gen  Ellen 
Pawlikowski,  Deputy  Director  of  the  NRO  stated,  “In  the  almost  decade  since  the  costly 
failures  of  the  late  1990s,  the  Air  Force  Space  and  Missile  Systems  Center  (SMC)  and  the 
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NRO  have  adopted  a  “back-to-basics”  approach  to  mission  assurance”  (Pawlikowski, 
2008).  These  changes,  while  intended  to  promote  mission  success  and  assured  access  to 
space,  also  had  the  unintended  consequence  of  increasing  costs  and  extending  launch 
schedules. 

In  order  to  drive  down  cost  and  schedule  impacts,  the  DoD  generated  a  new 
approach  to  space  launch.  Boeing’s  Delta  IV  and  Lockheed  Martin’s  Atlas  V  launch 
vehicles  became  the  crucible  of  our  Nation’s  space  launch  capability  for  large  DoD  and 
IC  satellites.  This  approach  became  known  as  the  Evolved  Expendable  Launch  Vehicle 
(EELV),  and  was  intended  to  solve  cost  and  schedule  issues  via  competitive  pricing  and 
assured  access  to  space  on  either  of  the  two  launch  vehicles  (Col  R.K.  Saxer  et  al,  2002). 

In  2005,  Boeing  and  Lockheed  Martin  fonned  the  United  Launch  Alliance 
(ULA);  the  organization  the  DoD  currently  uses  to  contract  Delta  IV  and  Atlas  V  launch 
support  (United  Launch  Alliance,  2013).  Moreover,  the  DoD  is  actively  pursuing  the 
possibility  of  commercial  space  launch  from  vendors  such  as  Space  X,  the  company  that 
has  recently  supported  multiple  resupply  missions  for  the  International  Space  Station 
(ISS). 

Neither  of  these  initiatives  has  shown  the  expected  success  or  even  a  significant 
reduction  in  cost  for  space  launch.  Despite  solid  efforts,  costs  for  space  launch  continue 
to  rise.  For  example,  the  cost  for  NASA’s  recently  scheduled  Atlas  V  launch  in 
November  2013  is  currently  estimated  at  $187  million  in  2010  dollars.  The  exact  same 
launch  capability  (NASA’s  MAVEN  mission)  was  contracted  three  years  earlier  for  the 
amount  of  $124  million  in  2007  dollars  (Clark,  2011). 
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1.2  Research  Objective 

The  authors  were  tasked  with  understanding  the  implications  of  the  space  launch 
process  on  satellite  acquisition  timelines,  with  the  intent  of  identifying  critical  aspects 
where  changes  may  reduce  schedule  delays.  Specific  direction  included  the  modification 
and  extension  of  an  existing  model  called  the  Enterprise  Requirements  and  Acquisition 
Model  (ERAM). 

1.3  Boundary  Conditions,  Assumptions,  and  Constraints 

The  Space  Launch  Model  was  developed  to  understand  delays  plaguing  a  space 
program  from  Milestone  C  through  launch.  Within  the  model  are  several  key 
assumptions  and  constraints.  First,  the  model  uses  data  only  from  EELV  missions,  since 
it  is  the  primary  launch  capability  for  the  United  States  DoD  and  NRO  satellites.  The 
data  was  collected  from  the  Launch  Information  Support  Network  (LISN),  and  includes 
missions  from  June  2006  through  March  2013.  Two  missions,  the  Defense  Satellite 
Communications  System  (DSCS)  IIIB-27  and  IIIB-06,  did  not  have  any  data  entered  into 
LISN  and  were  subsequently  ignored.  No  other  EELV  mission  data  was  available  in  the 
database. 

A  constraining  factor  related  to  LISN  is  that  the  data  was  not  captured  in  a 
standardized  timeframe  relative  to  each  program.  The  data  input  was  initiated  relative  to 
the  initial  launch  request  for  a  specific  mission  and  did  not  indicate  relative  timing  to  key 
program  dates  such  as  Milestone  C.  In  discussions  with  subject  matter  experts  (SMEs),  it 
was  found  this  timeline  could  vary  significantly  based  on  the  type  of  satellite  program. 
Spacecraft  maturity,  constellation  size,  complexity,  and  budget  certainty  all  impact  the 
time  it  takes  for  a  system  to  proceed  from  Milestone  C  to  launch.  As  an  example, 
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spacecraft  implementing  groundbreaking  technologies  and  capabilities  will  take  longer 
than  the  re-launch  of  already  proven  technologies,  or  multi-satellite  procurement  efforts 
such  as  the  Advanced  Extremely  High  Frequency  (AEHF)  and  GPS  programs. 

Differences  arise  due  to  launch  vehicle  types  and  configurations  as  well.  Atlas  V  and 
Delta  IV  Medium  launches  are  requested  2  years  from  the  planned  launch  date  while 
Delta  IV  Heavy  missions  are  requested  3  years  in  advance.  In  an  effort  to  simplify  this 
issue,  the  authors  elected  to  utilize  a  “black  box”  to  simulate  the  time  period  immediately 
following  Milestone  C  through  the  initial  request  for  Launch  Vehicle  support.  This 
method  allows  follow-on  users  to  input  the  originally  planned  time  for  Milestone  C  to 
launch.  For  the  purpose  of  this  report  all  simulations  were  run  with  this  “black  box” 
removed,  therefore  indicating  only  delay  time,  not  total  time  from  Milestone  C  to  launch. 

Due  to  the  inclusion  of  a  foreign  military  officer  in  the  research  group, 
classification  of  data  sources  presented  a  significant  constraint.  While  research  did 
include  generic  information  and  unclassified  mission  names,  it  could  not  include  specifics 
about  the  satellite  designs  for  either  DoD  or  NRO  missions.  This  constraint  was 
sufficiently  mitigated  by  leveraging  the  authors’  experience  in  space  acquisitions,  as  well 
as  significant  discussion  with  SMEs  to  generalize  and  interpret  sensitive  data. 

Initially,  the  research  focused  on  single-satellite  and  first-satellite-launched 
histories.  The  intent  was  to  better  understand  the  space  vehicle  schedule  implications 
associated  with  a  first-run  satellite  design.  Unfortunately  this  data  proved  impossible  to 
correlate,  as  classified  programs  do  not  indicate  whether  the  satellite  is  a  first  run  or 
reproduced  satellite  design.  Additionally,  the  low  number  of  satellite  programs  was 
deemed  a  concern  by  the  authors  and  the  researched  sample  set  was  increased  to 
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incorporate  timelines  for  programs  in  which  multiple  satellites  launched,  such  as  the 
Global  Positioning  System  (GPS)  ILF -2  and  IIF-3. 

Finally,  the  Extend  Sim  software  used  to  create  simulation  models  for  both 
ERAM  and  the  Space  Launch  process  was  unable  to  easily  replicate  a  “negative  delay”  or 
schedule  acceleration.  In  a  few  seldom  cases,  data  indicated  small  accelerations,  or 
“negative  delay”  in  portions  of  satellite  vehicle  (SV)  programs.  Due  to  Extend  Sim 
limitations,  this  data  was  modeled  as  “no  delay”  and  is  considered  to  have  a  negligible 
impact  on  the  simulation  results. 
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II.  Literature  Review 


Extensive  background  research  was  completed  to  understand  the  acquisition 
process,  specific  space-system  issues  related  to  acquisitions,  existing  models  of  the 
acquisition  process,  and  space-system  launch  processes.  In  excess  of  40  policy 
documents,  official  instructions,  journal  articles,  briefings,  and  acquisition  models  were 
reviewed  to  capture  the  space  acquisition  and  launch  processes.  Unfortunately,  no  single 
document  provided  a  concise  overview  or  model  of  the  space  launch  process.  The 
sources  referenced,  as  well  as  infonnation  gleaned  from  SMEs,  were  used  to  baseline  the 
model.  In  order  to  produce  the  Space  Launch  Model,  previous  models  of  the  acquisition 
process  were  reviewed.  ERAM  provided  the  modeling  foundation  of  this  effort  and  thus 
was  a  primary  piece  of  literature  reviewed.  Additional  significant  documents  included 
literature  dictating  the  Current  Launch  Schedule  Review  Board  (CLSRB)  process,  which 
is  used  to  manifest  all  US  launches.  The  history  of  launch  was  utilized  to  understand 
how  the  space  launch  process  has  evolved  and  is  continuing  to  evolve  in  the  current  space 
acquisition  environment. 

2.1  ERAM 

In  2008,  ERAM  was  developed  by  Lieutenant  Colonel  J.  Robert  Wirthlin  in  an 
effort  to  understand  key  interactions  between  the  Requirements  generation  (i.e.  Joint 
Capabilities  Integration  Development  System),  Funding  (i.e.  Planning,  Programming, 
Budgeting  and  Execution),  and  Acquisition  Program  portions  of  the  DoD’s 
organizational  acquisition  process  from  post-Milestone  A  through  Milestone  C. 

Wirthlin’s  model  focused  on  the  schedule  implications  of  the  process  and  interactions 
associated  with  each  arm  of  the  acquisition  process,  in  an  effort  to  identify  critical 
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interactions  that  regularly  led  to  significant  schedule  delays  and  proposed  process  or 
policy  modifications  that  could  be  implemented  to  reduce  schedules  for  acquisition 
programs.  Specifically,  Wirthlin  focused  on  three  primary  questions:  “How  does  the 
acquisition  system  work?”,  “Why  does  the  system  behave  the  way  that  it  does?”  and, 
“Are  there  things  that  can  be  done  to  improve  the  system?”  (Wirthlin,  2009).  Majors 
Leach  and  Searle  extended  ERAM  in  2010  focusing  specifically  on  space  system 
acquisition  efforts  (Leach  and  Searle,  2010).  Major  Montgomery  extended  ERAM  yet 
again  in  201 1  by  modeling  the  rapid  acquisition  process  often  used  by  organizations  such 
as  the  Special  Operations  Command  (SOCOM)  and  the  Rapid  Capabilities  Office  (RCO) 
(Montgomery,  2011). 

Due  to  the  scope  of  ERAM  it  only  modeled  acquisition  programs  in  the 
Technology  Development  and  Engineering  and  Manufacturing  Development  phases,  it 
did  not  model  acquisition  programs  from  Milestone  C  to  an  Initial  Capability, 

Operational  Capability,  or  in  the  case  of  this  research,  launch  of  a  satellite.  This  posed 
significant  problems  for  this  research,  as  space  program  data  is  not  consistently  captured 
or  is  captured  at  higher  classifications. 

2.2  Space  Launch  Policy 

The  Air  Force  Space  Command’s  (AFSPC)  launch  scheduling  process  guidance 
and  lower  echelon  documentation  was  reviewed  to  ensure  comprehensive  understanding 
of  the  manifesting  processes.  The  primary  document  which  provided  the  information 
necessary  to  understand  these  processes  was  Air  Force  Instruction  (AFI)  10-1211,  Space 
Launch  Operations .  AFI  10-1211  outlines  the  roles  and  responsibilities  of  the  Air  Force 
as  the  DoD  Executive  Agent  for  Space.  Furthermore,  it  places  the  SMC  Commander  as 


8 


the  sole  focal  point  for  certification  of  all  DoD  and  NRO  launch  vehicles.  Additionally, 
this  document  specifies  that,  “launch  schedule  execution  will  be  based  on  national 
priorities,”  and  designates  AFSPC  as  the  responsible  agent  for  establishing  the  manifest 
for  all  DoD,  Civil  and  commercial  missions  (Chandler,  2006). 

2.3  Current  Launch  Schedule  Review  Board  (CLSRB) 

The  launch  manifest  process  is  outlined  in  AFSPCI1 0-1 21 3AFSPCGM1,  Guidance 
Memorandum  (GM)  to  AFSPCI 10-1213,  Launch  Scheduling  and  Forecasting 
Procedures  and  Air  Force  Space  Command  Long  Range  Launch  Scheduling  Process 
(Weinstein,  2012,  LeMaitre,  2005).  These  documents  discuss  the  CLSRB  process  from 
the  initial  launch  support  request  through  launch  for  space  systems.  Specifically,  Air 
Force  Space  Command  Long  Range  Launch  Scheduling  Process  outlines  the  National 
Launch  Forecast  (NLF)  compilation  in  the  4-to-l  1  year  future  and  how  it  flows  into  the 
Space  Launch  Manifest  (SLM),  which  is  a  near-tenn,  three  year  schedule  for  launches. 
The  CLSRB  is  a  body  of  stakeholders  convened  biannually  to  certify  the  next  18  months 
of  the  SLM  (LeMaitre,  2005).  AFSPC110-1213  AFSPCGMl  implements  minor  changes 
to  the  process  by  creating  a  series  of  Launch  Commit  Reviews  (LCRs)  to  assess  risk 
related  to  launch  vehicle  (LV)  readiness,  space  vehicle  (SV)  readiness,  ground/control 
system  readiness,  and  operations  readiness.  It  further  delineates  responsibilities  between 
the  SMC  and  14th  AF  Commanders  for  each  of  these  risk  assessments,  and  assesses 
missions  scheduled  for  the  next  18  months  (Weinstein,  2012).  These  documents 
demonstrate  the  importance  of  understanding  launch  scheduling  with  respect  to  the 
number  of  space  missions  requiring  launch  and  the  capability  of  the  US  launch  industry. 
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2.4  Space  Launch  Assessments 

The  single  most  significant  document  related  to  evolution  of  the  space  launch 
process  over  the  past  1 5  years  is  the  Space  Launch  Vehicle  Broad  Area  Review  (SL  V 
BAR).  The  SLV  BAR,  led  by  Gen.  Larry  D.  Welch,  highlighted  several  problems  with 
the  space  launch  process  which  occurred  in  the  1990’s.  Specifically,  the  increase  in 
launch  failure  rates  from  1  per  year  over  a  12  year  period  to  5  failures  within  10  months. 
Mission  assurance  and  quality  incidents  also  raised  from  18  in  200  launches  to  9  in  51 
launches,  a  100%  increase  (US  Air  Force,  1999).  The  SLV  BAR  began  a  period  of  intense 
scrutiny  related  to  launch  vehicle  mission  assurance,  but  the  added  attention  to  detail  and 
slower  pace  yielded  strong  success  rates  according  to  the  article,  Assured  Access  to  Space 
in  a  Competitive  World  (Chilton,  2006).  RAND,  a  non-profit  research  and  analysis 
organization  intended  to  improve  policy  and  decision-making,  highlighted  additional 
issues  with  the  space  launch  segment,  discussing  the  ramifications  of  a  reduced 
commercial  launch  requirement  on  the  cost  and  schedule  of  government  launches.  These 
issues  ultimately  led  to  the  combination  of  the  Delta  IV  and  Atlas  V  teams  fonning  the 
United  Launch  Alliance  to  preserve  the  United  States  Evolved  Expendable  Launch 
Vehicle  (EELV)  heavy  lift  capability  (RAND  Corporation,  2006). 

2.5  Space  System  Acquisition  Delay  Assessments 

Multiple  Government  Accountability  Office  (GAO)  reports  were  reviewed  to 
assess  current  and  historical  space  system  schedules.  The  GAO  conducts  an  annual 
assessment  of  acquisition  programs.  The  “Defense  Acquisition  Assessment  of  Selected 
Weapons  Programs”  for  years  2006  -  2012  were  read  to  understand  how  space  program 
schedules  evolved  over  time  and  what  the  major  contributing  factors  were.  These  GAO 
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reports  repeatedly  highlighted  issues  with  technology,  design,  and  production  maturity 
for  the  spacecraft.  Other  issues  included  synchronization  of  space  and  ground  segment 
activation,  changes  in  prescribed  program  production  rates,  software-related  delays,  and 
fiscal  and  manning  constraints  (GAO,  2006-12). 

In  addition  to  a  review  of  the  GAO  reports,  the  Launch  Infonnation  Support 
Network  (LISN)  maintains  a  database  of  Launch  Change  Requests  submitted  after  an 
initial  request  for  launch  support.  This  database  was  queried  for  all  EELV  missions  and 
was  used  as  the  data  from  which  all  statistics  were  produced  (AFSPC,  2013). 
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III.  Methodology  and  Analysis 


3.1  Process 

Research  was  initiated  by  gaining  a  comprehensive  understanding  of  the  existing 
ERAM  model  created  by  Lt  Col  Wirthlin,  as  well  as  the  extensions  authored  by  Majors 
Searle,  Leach,  and  Montgomery.  A  literature  review  was  completed  to  determine 
existing  policies  relating  to  space  launch,  review  studies  documenting  the  space  launch 
processes  and  how  these  subjects  interact  within  the  overarching  DoD  acquisition 
process. 

The  first  major  decisions  determined  how  to  model  the  critical  portion  of  the 
launch  process.  The  team  assessed  the  space  acquisition  pre-Milestone  C  timeline  was 
sufficiently  modeled  in  the  Searle  and  Leach  extension  to  ERAM.  Therefore  the  team 
chose  to  focus  only  on  the  post-Milestone  C  timeline.  Additionally,  it  was  deemed 
outside  the  scope  of  this  research  project  to  model  the  post-Milestone  C  timeline  in  as 
much  intricate  detail  as  the  whole  of  ERAM.  Eventually,  the  decision  was  made  to 
model  only  the  delays  a  space  acquisition  program  is  likely  to  face  between  Milestone  C 
and  launch,  relative  to  the  planned  time  from  Milestone  C  to  launch.  Since  planned 
timelines  for  individual  programs  vary  greatly  post-Milestone  C,  focusing  only  on 
timeline  delay  greatly  simplified  the  modeling  process.  Additionally,  as  noted  by 
multiple  SMEs,  program  managers  were  more  interested  in  identifying  delays  their 
programs  may  incur  rather  than  another  model  of  their  already  planned  timeline.  The 
addition  of  the  “black  box”  mentioned  previously  will  allow  users  to  model  the  entire 
timeline  from  inception  to  launch,  if  desired.  This  will  also  allow  for  connection  of  the 
Space  Launch  Model  to  the  larger  ERAM  model. 
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The  team  collected  and  assessed  delays  related  to  the  space  launch  process 
associated  with  the  Atlas  V  and  Delta  IV  EELV.  The  authors  developed  a  simple  model 
of  the  space  launch  process  using  Grounded  Theory  Development  and  Inductive 
Reasoning  Methods  to  gain  further  insight  into  key  integration  and  decision  points  (Cal 
Poly,  2013). 


Figure  1:  Methodology  for  the  Development  of  the  Space  Launch  Process  Model 

Data  was  gathered  via  discussions  with  SMEs  from  the  SMC  at  Los  Angeles  Air 
Force  Base,  the  NRO  in  Chantilly,  Virginia,  and  other  space-community  locations 
throughout  the  United  States.  A  total  of  14  SMEs  were  utilized,  included  members  from 
the  acquisition  process,  Air  Staff,  the  user  community,  various  space  SPOs,  and  the  space 
launch  community.  Furthermore,  SME  volunteers  ranged  from  government  civilian, 
military,  support  contractors  from  the  Aerospace  Corporation,  a  Federally  Funded 
Research  and  Development  Corporation  (FFRDC),  and  industry  contract  partners  from 
the  ULA.  Most  SMEs  had  15  or  more  years  in  the  industry,  and  some  had  as  many  as  30 
years  experience  with  the  space  launch  process,  including  several  active  and  retired 
senior  military  leaders.  The  SME  discussions  covered  the  full  spectrum  of  the  space 
launch  process  to  include  space  launch  requirements,  budgeting,  space  vehicle 
integration,  and  launch  operations. 
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The  team  drafted  an  initial  description  model  based  on  author  experience  and 


relevant  literature  to  guide  discussions  with  SMEs.  It  was  quickly  determined  the  draft 
process  was  overly  complicated  and  inaccurate  in  several  areas.  This  led  to  a  simpler 
delay  based  model  focusing  on  SV-to-LV  integration  and  the  CLSRB  manifesting 
process.  Using  data  coding  techniques,  the  team  was  able  to  discern  seven  primary 
categories  of  delay  plaguing  the  space  launch  process  post-Milestone  C  (Johnson,  2013). 
Those  delay  categories  are  described  in  Table  1. 

Table  1:  Taxonomy  of  Space  Launch  Delay  Categories 


S] 

pace  Launch  Delay  Categories 

Delay  Type 

Description 

Space  Vehicle  -  Early 
(SV  Early) 

(>18  months) 

Delay  initiated  by  the  SV  program  office  18-months  or 
more  prior  the  predicted  launch  date.  These  delays 
typically  have  little  impact  on  the  ability  to  manifest  a 
specific  desired  launch  date  at  either  Vandenberg  AFB 
(VAFB)  or  Cape  Canaveral  Air  Force  Station  (CCAFS). 

Space  Vehicle  -  Late 
(SV  Late) 

(<18  months) 

Delay  initiated  by  the  SV  program  office  within  1 8- 
months  of  the  current  predicted  launch  date.  These 
delays  often  impact  the  ability  to  manifest  a  desired 
launch  date  at  either  Vandenberg  AFB  or  Cape 

Canaveral  AFS,  depending  on  manifest  density.  The  SV 
late  delays  were  often  of  shorter  duration  than  SV  early 
delays,  leading  to  a  separate  distribution. 

Launch  Vehicle  -  Long 
Tenn  (LV  Long) 

(>18  months) 

Delay  initiated  by  the  launch  vehicle  or  associated 
leadership  due  to  known  manufacturing  issues,  launch 
separation  requirements,  or  updates  to  an  Initial  Launch 
Capability  (ILC)  for  the  specific  mission.  These  delays 
typically  have  little  impact  on  the  ability  to  manifest  a 
specific  desired  launch  date  at  either  Vandenberg  AFB 
or  Cape  Canaveral  AFS. 

Launch  Vehicle  -  Short 
Tenn 

(LV  Short) 

(<18  months) 

Delay  initiated  by  the  launch  vehicle  or  associated 
leadership  due  to  unforeseen  issues  with  the  launch 
vehicle,  near-tenn  launch  date  change  requests  by  the 
mission  integrator,  or  a  launch  vehicle  anomaly  on  a 
previous  mission  that  has  a  ripple  effect  on  the  mission 
of  interest.  These  delays  may  impact  the  ability  to 
manifest  a  desired  launch  date  at  either  Vandenberg  AFB 
or  Cape  Canaveral  AFS,  depending  on  manifest  density. 
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Re-queue 

Delay  or,  in  seldom  cases,  acceleration  encountered 
when  a  program  attempts  to  re-enter  the  launch  manifest 
after  it  was  removed  due  to  another  delay  such  as  SV- 
Early.  This  occurs  more  often  as  the  re-entry  attempt  is 
closer  to  the  planned  launch  date,  generally  within  1 8 
months. 

Priority 

Delay  or  acceleration  of  the  launch  date  due  to  mission 
priorities.  This  occurs  when  the  CLSRB  process  or 
senior  leadership  determines  a  launch  date  must  slip  or 
in  seldom  cases  move  earlier  to  accommodate  mission 
requirements. 

Weather  /  Miscellaneous 
(Wx  /  Misc.) 

Delay  of  relatively  short  duration  caused  by  weather, 
launch  window  refinement,  or  launch  range  support 
issues. 

After  review  of  the  various  individual  delays  space  acquisition  programs  incurred 
relative  to  planned  timelines,  patterns  began  to  develop.  Delays  either  occurred  for 
similar  reasons,  at  similar  times  within  the  planned  timeline  or  were  due  to  common 
external  factors.  This  led  to  the  creation  of  the  above  common  and  general  categories. 
These  categories  aided  in  simplifying  the  following  model  and  provided  a  venue  for  later 
analysis.  Most  significantly,  data  coding  ensured  individual  categories  fit  somewhat 
common  and  manageable  distributions  for  inclusion  into  the  resulting  model. 

Once  the  high-level  categories  were  identified  and  defined,  historical  data  was 
collected  from  the  LISN  database  maintained  by  the  Launch,  Ranges  and  Networks 
Division  of  Air  Force  Space  Command  Headquarters.  Data  was  collected  from  the  LISN 
database  by  searching  for  all  previous  EELV  missions,  which  resulted  in  data  availability 
from  June  2006  through  March  2013.  In  total,  the  search  resulted  in  35  Atlas  V  and 
Delta  IV  missions  (16  Delta  IV  and  19  Atlas  V).  Two  missions,  DSCS  IIIB-27  and  IIIB- 
06,  didn’t  have  any  data  entered  into  LISN  and  were  subsequently  ignored.  The  final 
dataset  included  33  missions  with  a  complete  historical  record  of  launch  date  changes  and 
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causes  for  these  changes.  Each  mission  history  yielded  many  Launch  Change  Request 
(LCR)  data  inputs,  ranging  from  as  few  as  four  LCRs  to  as  many  as  32  LCRs.  The  team 
then  binned  the  individual  delays  found  on  LISN  into  the  most  appropriate  category,  and 
separated  them  by  launch  vehicle  type  (Atlas  vs.  Delta)  to  allow  for  statistical  analysis. 

As  mentioned  previously,  SME  discussions  indicated  a  potential  difference 
between  the  space  launch  timelines  associated  with  Atlas  and  Delta  missions. 
Specifically,  it  was  believed  the  Launch  Vehicle  Long  Tenn,  Launch  Vehicle  Short 
Tenn,  Re-queuing,  and  possibly  the  Priority  delays  were  dependent  upon  the  launch 
vehicle  type  and  associated  reliability  and  launch  rates.  Delays  related  to  the  space 
vehicle,  both  early  and  late,  as  well  as  weather  and  miscellaneous  delays  were  expected 
to  be  launch  vehicle  agnostic. 

Microsoft  Excel  was  used  to  consolidate  and  manipulate  data.  LCRs  were 
reviewed  and  manually  grouped  into  one  of  the  seven  delay  category  bins.  Once  all 
mission  data  was  binned,  each  delay  category  was  assessed  to  detennine  whether  a 
statistical  difference  existed  between  delays  associated  with  the  Atlas  V  and  Delta  IV 
launch  vehicles.  The  statistics  gathered  for  each  of  these  groups  of  data  included  the 
mean,  median,  mode,  minimum,  maximum,  and  standard  deviation  of  the  average  delays 
by  delay  category,  as  shown  in  Table  2. 

Upon  examination  of  the  data,  the  research  team  found  the  delay  categories  did 
not  appear  significantly  different  based  on  the  launch  vehicle  type;  a  direct  contradiction 
to  expectations  from  SME  discussions.  A  t-test  was  completed  against  the  null 
hypothesis  that  the  Atlas  and  Delta  sample  means  were  equal  for  each  factor.  The  t- 
statistic  was  calculated  using  the  average  length  of  delay  in  each  category  for  a  specific 
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2 

LV,  Xn,  the  number  of  delays,  n,  and  the  variance  of  those  delays,  S'.  The  degrees  of 


freedom  were  calculated  using  the  Smith-Satterthwaite  method  of  calculation  (Milton  & 


Arnold,  2003).  The  t-statistic  and  Smith-Satterthwaite  Degrees  of  Freedom  equations  are 
shown  below: 


t  -  statistic 


(Xl-X2)-(m-q2) 


[S;hh+S?ln2f 
[Sf/ntf  ,  [S22/n2f 
n,  - 1  n2  - 1 


(1) 

(2) 


Table  2:  Initial  Statistics  of  Launch  Delay  Categories 


Atlas  Delay  Category  Statistics  (Months) 

SV-Early 

SV-Late 

LV-Long 

LV-Short 

Re-queue 

Priority 

Wx/Misc 

Sample  Size 

25 

83 

32 

28 

31 

15 

17 

Probability 
of  Delay 
within 
Category 

0.58 

0.89 

0.63 

0.68 

0.68 

0.47 

0.42 

Average 

Delay 

3.77 

1.18 

1.22 

0.50 

1.75 

1.87 

0.14 

Median 

3.03 

0.46 

0.64 

0.25 

0.43 

Std  Dev 

3.99 

2.84 

2.62 

0.63 

2.20 

4.73 

Min  Delay 

-3.09 

-5.46 

-1.22 

0.03 

-0.10 

Max  Delay 

10.99 

15.33 

14.05 

2.57 

6.97 

12.99 

0.72 

Delta  Delay  Category  Statistics  (Months) 

SV-Early 

SV-Late 

LV-Long 

LV-Short 

Re-queue 

Priority 

Wx/Misc 

Sample  Size 

20 

24 

41 

28 

24 

11 

10 

Probability 
of  Delay 
within 
Category 

0.58 

0.89 

0.63 

0.68 

0.68 

0.47 

0.42 

Average 

Delay 

4.40 

1.09 

1.85 

0.62 

2.30 

2.35 

0.24 

Median 

2.99 

0.81 

1.22 

0.12 

1.25 

2.04 

0.05 

Std  Dev 

7.68 

1.33 

2.35 

1.64 

2.85 

3.26 

0.52 

Min  Delay 

-1.02 

-0.95 

-4.05 

0.03 

-0.30 

-2.53 

0.03 

Max  Delay 

35.00 

4.18 

8.98 

8.75 

11.18 

8.98 

1.71 
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Due  to  the  hypothesis  structure  of  this  specific  t-test,  any  p-values  above  0.05 
indicate  the  null  hypothesis  cannot  be  rejected,  and  therefore  the  data  is  assumed  to  have 
similar  means.  The  p-values,  shown  in  table  3,  are  all  greater  than  0.05,  and  demonstrate 
that  the  Atlas  and  Delta  average  delays  cannot  be  distinguished  from  one  another,  at  a 
0.05  level  of  significance.  Results  of  t-test  comparison  are  shown  in  Table  3. 


Table  3:  T-Test  of  Atlas  and  Delta  Delay  Categories 


SV-Early 

SV-Late 

LV-Long 

LV-Short 

Re-queue 

Priority 

Wx/Misc 

T  Statistic 

-0.3305 

0.2197 

-1.0600 

-0.3645 

-0.7885 

-0.3094 

-0.5674 

P-Value 

0.7436 

0.8266 

0.2932 

0.7177 

0.4349 

0.7597 

0.5818 

Based  on  the  results  above,  the  first  significant  finding  is  that:  There  is  no 
significant  statistical  difference  in  the  mission-level  space  launch  delays  associated  with 
the  Delta  IV  and  Atlas  V  launch  vehicles.  Based  on  the  finding  that  no  statistical 
difference  existed  between  EELV  launch  vehicle  types,  the  data  was  consolidated  into  a 
single  grouping  for  all  further  analysis,  as  shown  in  Table  4. 

The  team  then  determined  a  plan  to  model  the  individual  delays  within  the  Extend 
Sim  discrete  modeling  tool.  The  goal  was  to  create  an  accurate  simulation  while  focusing 
on  simplicity  and  minimizing  the  introduction  of  error.  The  decision  was  made  to  model 
each  of  the  seven  delay  categories  in  series  using  a  double  loop  for  each  delay  type.  All 
delays  were  modeled  separate  from  of  one  another.  An  example  is  shown  in  Figure  2. 

The  seven  individual  delays  were  programmed  within  Extend  Sim  in  series  as  an  entire 
simulation  segment,  as  shown  in  Figure  3. 
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Table  4:  Overall  Launch  Delay  Statistics 


Overall  Launch  Delay  Statistics  for  33  Programs  (Months) 

SV-Early 

SV-Late 

LV-Long 

LV-Short 

Requeue 

Priority 

Wx/Misc 

Sample  Size 

45 

107 

73 

56 

55 

26 

27 

Probability  of 
Delay  Within 

Category 

0.51 

0.73 

0.67 

0.61 

0.67 

0.58 

0.42 

Average  Delay 

4.05 

1.16 

1.57 

0.56 

1.99 

2.07 

0.18 

Median 

3.03 

0.49 

0.72 

0.21 

0.82 

1.00 

0.03 

Std  Dev 

5.85 

2.58 

2.47 

1.23 

2.49 

4.10 

0.35 

Min  Delay 

-3.09 

-5.46 

-4.05 

0.03 

-0.30 

-4.38 

-0.03 

Max  Delay 

35.00 

15.33 

14.05 

8.75 

11.18 

12.99 

1.71 
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Figure  2:  Example  of  Individual  Delay  Category  Loop 


Figure  3:  Model  as  Depicted  in  Extend  Sim 
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Figure  4:  Histograms  of  Individual  Delay  Category  Occurrences 

The  initial  probability  block  setting  for  each  category  was  determined  by  dividing 
the  number  of  programs  that  experienced  the  particular  delay  by  the  total  number  of 
programs.  This  determined  the  probability  a  particular  program  would  initially  delay 
within  a  particular  category.  This  calculation,  shown  below,  was  derived  using  data 
found  in  Table  5. 
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P initial  =  #  of  programs  delayed  in  category  /  #  of  total  programs 


(3) 


If  a  program  did  encounter  an  initial  delay  the  second  probability  block 
detennined  the  probability  of  additional  delays  within  the  same  category.  This 
probability  was  calculated  by  dividing  the  average  number  of  additional  delays  (average 
number  of  total  delays  per  program  per  category,  minus  1)  into  the  number  “1”,  and 
subtracting  this  quotient  from  “1”.  This  calculation  is  shown  below  and  was  derived 
using  data  found  in  Table  5.  This  approximation  was  accurate  when  the  average  number 
of  additional  delays  was  greater  than  one.  In  the  two  cases  when  it  wasn’t,  Priority  and 
Wx  /  Misc,  the  team  used  experience  to  estimate  the  probability  an  additional  delay 
would  occur.  This  structure  established  the  secondary  loop  which  determines  how  many 
times  a  particular  program  will  experience  a  particular  delay. 

P  additional  =  1  -  {1  /  [(avg  #  of  delays  per  program  per  category)  -  1] }  (4) 

The  actual  time  delays  themselves  were  simulated  in  Extend  Sim  using  activity 
blocks.  The  activity  blocks  simulate  a  time  delay  via  a  randomly  seeded  sampling  of  a 
pre-assigned  distribution.  Each  pass  through  the  activity  block  simulates  an  individual 
delay  occurrence,  and  then  flows  into  the  additional  probability  block  to  determine  if 
another  iteration  of  the  delay  will  occur.  If  not,  it  will  flow  on  to  the  next  delay  category 
loop,  as  depicted  above  in  Figure  2. 
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Table  5:  Overall  Delay  Occurrence  Statistics 


Delay  Occurrence  Probability  Data 

SV-Early 

SV-Late 

LV-Long 

LV-Short 

Requeue 

Priority 

Wx/Misc 

#  of  “No  Delay” 

16 

9 

11 

13 

11 

14 

19 

Total  Programs 

33 

33 

33 

33 

33 

33 

33 

Probability  of  First 
Delay  (Pinitial) 

0.51 

0.73 

0.67 

0.61 

0.67 

0.58 

0.42 

1  ”  P  initial 

0.49 

0.27 

0.33 

0.39 

0.33 

0.42 

0.58 

(Average  of 
“Non-0”  Delays)  -  1 

1.64 

3.45 

2.31 

1.80 

1.50 

0.36 

0.92 

Probability  of 
Additional  Delays 

(P  additional) 

0.61 

0.29 

0.43 

0.56 

0.67 

0.85 

0.75 

The  researchers  chose  the  proper  distributions  for  each  activity  block  after 
analysis  of  the  actual  program  delay  data.  Statistical  analysis  was  conducted  on  each 
delay  category’s  actual  data.  These  statistics  are  posted  above  in  Table  5.  Histograms 
were  created  detailing  the  time  distribution  of  actual  delay  data  per  category.  Probability 
density  functions  (PDFs)  were  chosen  to  best  approximate  the  histogram  data.  In  most 
cases  the  Inverse  Gaussian  function  most  closely  approximated  the  delay  data.  Microsoft 
Excel  add-in  “Solver”  was  used  to  minimize  the  cumulative  squared  error  between  the 
histogram  data  and  the  distribution  by  optimizing  the  Inverse  Gaussian  parameters,  a  and 
p.  The  following  equation  describes  this  technique.  Where  a  and  P  are  input  parameters 
to  the  Inverse  Gaussian  function,  F  is  the  Cumulative  Distribution  Function  (CDF) 


evaluated  at  the  corresponding  histogram  point  i,  FT  is  the  particular  histogram  value  and 
N  is  the  total  number  of  histogram  points. 


m,T1(a,  P)  l 


(5) 
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Two  particular  delay  category  histograms,  “SV  -  Early”  and  “Priority,”  did  not  closely  fit 
an  Inverse  Gaussian  distribution,  or  any  other  distribution  available  in  Extend  Sim.  In 
these  cases  simple  triangular  distributions  were  used,  with  minimum,  maximum  and  most 
likely  values  set  by  observation,  excluding  statistical  outliers  (Evans,  2000).  The 
distribution  parameters  used  in  the  model  are  shown  in  Table  6  below.  The  histograms 
for  each  delay  category  along  with  the  selected  overlying  distributions,  in  both  PDF  and 
CDF  format  are  shown  below  in  Figure  5. 

Table  6:  Delay  Category  Distribution  Parameters 


Distribution  Type 

Inverse  Gaussian 

Triangular 

Delay 

a 

P 

Minimum 

Maximum 

Most  Likely 

SV  -  Early 

-3 

12 

4 

SV  -  Late 

1.2 

3 

LV  -  Long  T erm 

1.1 

1.8 

LV  -  Short  T erm 

0.01 

0.1 

Re-queue 

0.59 

4.97 

Priority 

-4 

10 

1.3 

Wx  /  Misc 

0.152 

0.159 
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Figure  5:  Delay  Category  Histograms  with  Distributions  in  PDF  and  CDF  Format 


24 


In  order  to  determine  the  number  of  Monte  Carlo  replications,  R,  required  to 
achieve  the  relative  precision,  s,  of  .05,  at  a  significance  level,  a,  of  .05,  the  team 
employed  the  following  equation.  S0  is  the  sample  standard  deviation  relative  to  the 
mean.  The  degrees  of  freedom,  df,  are  equal  to  32,  one  less  than  the  sample  size  of  33 
programs.  This  calculation  references  the  Student  T  distribution. 

R  =  (Ta/df)2S02/s2  (6) 

From  this  calculation,  it  was  found  that  473  replications  were  required.  Therefore 
all  experiments  were  completed  using  1000  simulations,  based  on  the  simulation  time 
available  and  the  relative  precision  achieved. 

3.2  Verification 

The  team  verified  the  model  by  comparing  two  criteria  between  actual  and 
modeled  data.  The  first  element  compared  was  the  average  number  of  times  a  program  or 
simulation  experienced  the  individual  delay  categories.  This  assessment  was  used  to 
verify  the  accuracy  of  the  probability  blocks  used  to  simulate  delay  occurrences,  and  is 
shown  below  in  Table  7  below.  In  each  delay  category  the  model  simulations 
experienced  fewer  occurrences  on  average  than  the  actual  programs.  This  difference 
ranged  from  16-41%  amongst  the  delay  categories.  The  team  considered  this  difference 
substantial,  but  accepted  it  as  within  reasonable  error  bounds  given  the  variance  of  the 
sample  data  and  relative  accuracy  of  the  overall  simulation  results. 

While  the  method  used  was  statistically  sound,  it  most  likely  underestimated  the 
number  of  delay  occurrences  because  multiple  delays  were  simulated  using  only  two 
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probabilities,  P initial  and  P additional.  This  technique  was  used  to  simulate  the  potential  for 
many  delays,  while  minimizing  overall  complexity  and  maximizing  flexibility  within  the 
model.  The  error  had  a  “delay  shortening”  affect  on  the  overall  model  results. 
Additionally,  Inverse  Gaussian  distributions  were  used  to  simulate  most  of  the  actual 
delay  categories.  This  caused  an  unrealistically  long  delay  to  be  occasionally,  but  rarely, 
encountered  due  to  the  infinitely  long  “tail”  of  the  distribution.  Regardless  of  this  “delay 
lengthening”  affect,  Inverse  Gaussian  distributions  still  provided  the  best  fit. 

The  team  assessed  the  combination  of  the  above  discussed  “delay  shortening” 
occurrence  estimation  error  and  the  “delay  lengthening”  distribution  error  actually 
combined  to  form  an  accurate  end  result.  For  these  reasons,  the  team  chose  to  accept 
these  errors.  The  team  recommends  future  work  focusing  on  reducing  or  negating  these 
errors  altogether  in  future  models. 

Table  7:  Model  vs.  Actual  Delay  Occurrences 


Comparison  of  Model  vs.  Actual  Delay  Occurrences 

SV-Early 

SV-Late 

LV-Long 

LV-Short 

Requeue 

Priority 

Wx/Misc 

Actual  Avg  #  of 
Delays  per  Prgm 

1.36 

3.24 

2.21 

1.70 

1.67 

0.79 

0.82 

Model  Avg  #  of 
Delays  per  Prgm 

0.80 

2.49 

1.56 

1.20 

0.99 

0.66 

0.58 

Difference 

41% 

23% 

29% 

29% 

41% 

16% 

29% 

The  overall  simulation  results  were  compared  by  basic  statistical  methods  as  well 
as  the  Kolmogorov-Smirnov  (KS)  test  for  like  distributions.  Results  of  the  basic  statistics 
are  shown  below  in  Table  8.  The  average  program  delay  between  actual  and  modeled 
data  is  within  1%  with  a  standard  deviation  of  4%.  The  maximum  result  of  the  model  is 
34%  longer  than  the  actual  data;  this  result  was  expected  due  to  the  cumulative  usage  of 
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Inverse  Gaussian  functions  and  the  “infinite,  but  highly-unlikely  tails”  Gaussian 
distributions  exhibit. 


Table  8:  Model  vs.  Actual  Delay  Statistics 


Overall  Comparison  of  Model  vs.  Actual  Program  Delay  in  Months 
(1000  Simulations  vs.  33  Programs) 

Min 

Max 

Average 

Std  Dev 

Model  Results 

0.00 

66.79 

18.62 

11.58 

Actual  Results 

0.69 

44.11 

18.82 

12.08 

Difference 

N/A 

34% 

1% 

4% 

The  histograms  of  the  model  and  actual  data  provide  a  visual  comparison  of  the 
respective  distributions.  Both  distributions  appear  to  display  a  bi-modal  nature.  Actual 
data  appears  to  have  modes  at  roughly  2  and  20-30  months,  while  the  model  outputs 
modes  at  2  and  10-20  months,  albeit  with  a  larger  tail.  Possible  reasons  for  this  bi-modal 
nature  are  discussed  in  the  following  Results  section.  The  model  and  actual  program 
delay  histograms  are  shown  below  in  Figure  6. 

Finally,  a  two-sample  KS  test  was  performed  to  test  for  like  distributions  between 
model  and  actual  data.  A  KS  test  is  a  nonparametric  test  used  to  compare  a  sample’s 
empirical  distribution  function  (EDF)  to  a  reference  CDF.  The  KS  test  is  sensitive  to 
differences  in  both  the  location  and  shape  of  the  concerned  distributions  (Justel,  Pena,  & 
Zamar,  1997).  In  this  case  the  simulated  data  was  treated  as  the  reference  CDF.  The 
actual  overall  program  delay  data  was  the  sample  EDF.  The  KS  test  statistic  quantifies 
the  distance  between  the  EFD’s  of  both  samples.  The  null  distribution  of  the  test  statistic 
is  calculated  under  the  null  hypothesis  that  both  samples  are  drawn  from  the  same 
distribution. 
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The  KS  test  was  run  as  a  function  within  MATLAB  software.  The  significance 
level  was  set  at  0.05.  The  calculated  test  statistic  was  0.1677,  with  a  p-value  of  0.3019. 
The  result  of  this  test  failed  to  reject  the  null  hypothesis  that  both  samples  are  from  the 


same  distribution.  The  graphical  result  of  the  KS  test  is  shown  below  in  Figure  7. 
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Figure  6:  Model  and  Actual  Histograms  of  Total  Program  Delay 
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Figure  7:  Results  of  KS  Test  for  Like  Distributions 
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IV.  Results 


In  general  the  model  shows  an  average  delay  beyond  the  program  baseline 
schedule  of  18.62  months.  The  delay  most  often  experienced  was  19  months,  and  75%  of 
programs  experienced  delays  less  than  25  months.  Table  9  below  shows  the  distribution 
of  expected  delays  broken  down  by  percentiles. 


Table  9:  Expected  Delay  By  Percentile 


Expected  Delay  vs.  Percentile  of  Overall  Programs  (Months) 

25% 

50% 

75% 

90% 

95% 

99% 

Expected  Delay 

<10.8 

<17.0 

<24.7 

<34.4 

<41.0 

<54.6 

As  noted  previously,  the  first  finding  was  that  no  difference  exists  between  Altas 
V  and  Delta  IV  launch  vehicles  with  respect  to  schedule  delays.  This  conclusion  was 
based  on  statistical  data  collected  from  33  historic  launch  schedules.  While  this  initially 
surprised  the  research  team,  the  data  shows  that  delays  within  each  of  the  seven  delay 
categories  can  vary  significantly  for  both  vehicles  and  result  in  data  groups  that  cannot  be 
statistically  separated.  However,  several  trends  were  noted  in  the  types  of  schedule 
delays. 

Based  on  the  delay  category  bins,  LCRs  submitted  within  those  bins,  and  SME 
experience  there  are  several  issues  that  are  often  associated  with  schedule  delays.  The 
issues  most  often  noted  are  multi-satellite  procurements,  varying  technology  risks, 
varying  satellite  complexity,  contractor  risk,  and  the  confidence  associated  with  original 
programmatic  schedule  estimates.  Often,  individual  delays  are  unforeseeable  and  are 
caused  by  manufacturing  issues  or  issues  associated  with  another  mission.  A  specific 
example  is  the  Delta  IV  In-Flight  Anomaly  that  occurred  on  the  GPS  IIF-3  that  impacted 
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the  launch  timeline  of  the  Orbital  Test  Vehicle  (OTV-3)  and  a  Space  Based  Infra-Red 
System  (SBIRS)  GEO-2  spacecraft  (Headquarters  Air  Force  Space  Command,  2013). 

As  previously  discussed,  historical  data  also  appears  to  exhibit  some  bi-modality. 
The  team  hypothesizes  that  this  bimodality  may  be  associated  with  a  program’s 
technology  maturity,  complexity,  whether  a  specific  mission  is  a  first-of-a-kind  or  a 
multi-satellite  constellation  replenishment  effort  or  whether  there  is  significant  urgency 
associated  with  a  specific  program.  A  relatively  small  number  of  programs  cause  the 
short-delay  mode  at  approximately  2  months,  and  the  experience  of  the  SMEs  has  shown 
that  programs  even  have  the  ability  to  accelerate  if  there  is  significant  impetus  and  close 
coordination  between  all  components  of  the  acquisition  process. 

The  average  program  experienced  a  delay  of  18.82  months,  but  there  appears  to 
be  a  great  variation  dependent  on  the  type  of  satellite  program.  Technologically  mature 
programs  face  fewer  threats  to  a  schedule  and  therefore  tend  to  have  fewer  slips  in  the 
integration,  testing,  and  launch  phases.  In  comparison,  those  space  systems  with 
significant  complexity,  either  technological  or  integrative  in  nature,  tend  to  experience 
significant  delays  in  integration,  testing,  and  launch. 

Urgency  of  need  has  shown  the  ability  to  drive  a  program  closer  to  an  estimated 
schedule.  This  is  primarily  because  Congressional  and  DoD  leadership,  the  PPBE 
process,  and  the  Government  and  Contractor  acquisition  processes  are  sometimes  able  to 
fonn  a  very  tight  team  when  national  security  is  at  stake.  While  this  situation  has  been 
experienced  by  one  of  the  authors  of  this  paper,  it  is  somewhat  abnormal  and  occurs  only 
when  the  direst  of  needs  arises.  Alternatively,  basic  constellation  replenishment  missions 
may  have  a  higher  tendency  of  being  artificially  delayed  if  the  current  constellation  is 
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healthy.  This  priority  shift  then  allows  launch  effort  to  focus  on  less  healthy  or  higher 
priority  programs. 

The  delay  category  appearing  to  have  the  most  significant  impact  on  a  program’s 
schedule  is  the  “Space  Vehicle  -  Early”  delay.  As  previously  described,  the  Space 
Vehicle  -  Early  delay  is  initiated  by  the  SV  program  office  18-months  or  more  prior  to 
the  predicted  launch  date.  At  this  point  in  an  acquisition  program  significant  fixes  or 
changes  may  be  incurred,  usually  extending  schedules  due  to  satellite  disassembly, 
reassembly,  test,  and  analysis  involved  in  the  specific  resolution.  While  this  delay  occurs 
on  average  only  1.36  times  per  program,  its  overall  time  is  the  largest  at  4.05  months  per 
delay. 

Alternatively,  the  team  hypothesizes  that  satellite  assembly,  integration  and  test 
issues  occurring  late  in  a  program  have  a  significant  time  impact  due  their  frequency  of 
occurrence  and  “ripple”  they  induce  in  the  overall  launch  process.  “Space  Vehicle  - 
Late”  delays  occur  on  average  3.24  times  per  program  but  only  incur  1.16  months  per 
delay.  More  importantly,  any  delay  within  18  months  of  planned  launch  has  the  potential 
to  induce  delays  in  another  delay  category,  such  as  Priority  or  Re-Queue.  This  ripple 
effect  can  be  largely  eliminated  if  the  initial  SV  delay  is  eliminated.  Additionally,  from  a 
systems  engineering  perspective  unforeseen  issues  late  in  a  program  tend  to  have  much 
greater  impact  on  cost  and  schedule  than  early  delays. 

The  contribution  of  overall  delay  and  occurrence  by  all  seven  delay  categories  is 
shown  in  Table  10  below.  As  mentioned  above,  SV  Early  delays  contribute  the  most  to 
the  overall  delay  a  program  can  expect  to  encounter,  at  29.3%.  In  fact,  as  seen  in  Table 
1 1  below,  roughly  half  of  all  program  delay  (49.3%)  is  encountered  due  to  the  SV 
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program  itself,  while  the  other  half  is  contributed  by  the  launch  vehicle  or  process. 
Additionally,  it  could  be  argued  the  Re-queue  delay  is  in  some  cases  due  to  a  late  SV 
program  delay,  hence  increasing  delay  contributed  by  the  SV  program. 

Table  10:  Overall  Delay  Category  Statistics 


Overall  Delay  Category  Statistics 

SV-Early 

SV-Late 

LV-Long 

LV-Short 

Re-queue 

Priority 

Wx/Misc 

Avg  #  of  Occurrences 
per  Program 

1.36 

3.24 

2.21 

1.70 

1.67 

0.79 

0.82 

%  Based  on 

Occurrence 

11.6% 

27.5% 

18.8% 

14.4% 

14.1% 

6.7% 

6.9% 

Avg  Time  Delay  per 
Occurrence  (Months) 

4.05 

1.16 

1.57 

0.56 

1.99 

2.07 

0.18 

%  Based  on  Average 
Time  per  Occurrence 

35.0% 

10.0% 

13.6% 

4.9% 

17.2% 

17.9% 

1.5% 

Avg  Total  Time 
Delayed  per  Program 
(Months) 

5.52 

3.77 

3.48 

0.95 

3.32 

1.63 

0.14 

%  Time  Delayed  per 
Program 

29.3% 

20.0% 

18.5% 

5.1% 

17.6% 

8.7% 

0.8% 

Table  11:  Consolidated  Delay  Statistics 


Consolidated  Delay  Statistics 

SV-Early 

SV-Late 

LV-Long 

LV-Short 

Re-queue 

Priority 

Wx/Misc 

Satellite  Vehicle 

Launch  Vehicle  or  Process 

%  Time  Delayed  per 
Program 

49.4% 

50.6% 

Satellite  Vehicle 

Launch  Vehicle 

Launch  Process  or  Range 

%  Time  Delayed  per 
Program 

49.4% 

23.5% 

27.1% 

Satellite  Vehicle 

Launch  Vehicle 

Launch  Process 

Wx/Misc 

%  Time  Delayed  per 
Program 

49.4% 

23.5% 

26.3% 

0.8% 
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V.  Conclusion  and  Recommendations 


The  research  team  identified  several  recommendations  for  the  space  acquisition 
community.  Based  on  SME  discussions,  current  scheduling  tools  are  fairly  accurate; 
however  the  perception  is  that  a  realistic  program  schedule  often  dooms  the  program  in 
tenns  of  support.  It  is  often  assumed  a  realistic  schedule  may  “scare”  approving  officials 
and  potentially  not  receive  funding  and  support.  This  leads  to  “green-light”  schedules  in 
an  effort  to  compete  with  other  programs  for  scarce  resources.  Ultimately,  the  original 
schedules  may  only  be  achieved  if  every  aspect  of  the  program  goes  smoothly,  including 
the  incorporated  schedule  margin.  Often,  a  “green  light”  schedule  is  impossible  even 
with  perfect  execution.  Based  on  SME  discussions  and  historical  data,  the  probability  of 
meeting  a  green-light  schedule  is  practically  impossible.  The  acquisition  community 
must  figure  out  how  to  overcome  this  culture.  Four  recommendations  are  provided 
below. 

1)  Programs  can  improve  schedule  estimates  by  utilizing  current  scheduling  methods 
and  incorporating  an  estimated  delay  within  schedule  margins.  This  delay  should  be 
based  on  the  estimated  average  delay  of  18.62  months,  but  increased  or  decreased  based 
upon  program-specific  factors.  These  factors  include  learning  curve  incorporation  of  past 
lessons  based  on  multi-satellite  procurements,  technology  risks,  satellite  complexity, 
contractor  risk,  and  the  general  risk  posture  associated  with  original  schedule  estimates. 
For  instance,  NRO  satellites  often  push  technological  capabilities  and  are  typically  more 
complex  than  DoD  communications  satellites.  As  an  example,  a  complex  NRO  satellite 
may  be  expected  to  have  a  longer  schedule  delay  between  Milestone  C  and  launch  than  a 
DoD  communications  satellite  has,  such  as  Mobile  User  Objective  System  (MUOS). 
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2)  Program  Managers  and  schedule  analysts  should  use  system  specific  criterion 
such  as  Technology  Readiness  Levels  (TRLs),  relative  experience  level  within  the  SPO 
and  overall  program  scale  to  help  estimate  program  timelines  more  accurately.  Given 
these  factors,  a  good  starting  estimate  for  program  delay  could  range  from  10  to  25 
months,  based  on  previously  calculated  delays  for  the  middle  50%  of  programs. 

3)  In  concert  with  increasing  schedule  margins  to  account  for  expected  schedule 
delays,  space  programs  should  continue  to  assess  a  “green-light”  schedule.  Similar  to  the 
“will  cost”  vs.  “should  cost”  estimation  approach  implemented  across  DoD  acquisition, 
satellite  acquisition  offices  should  consider  implementing  two  schedules,  a  “green-light” 
and  “most-likely”  schedule  for  senior  leadership  awareness  (Tang,  20).  Acquirers  should 
vigorously  pursue  the  “green-light”  schedule  with  satellite  contractors;  however, 
leadership  at  all  levels  should  be  aware  that  these  schedules  are  often  unobtainable  and 
that  the  “most-likely”  schedule  will  best  suit  planning  purposes  for  budgetary  and 
requirements  discussions. 

4)  Finally,  the  DoD  and  NRO  should  implement  better  practices  for  tracking 
historical  program  timelines  and  associated  causes  of  delay.  This  data  should  be  used  to 
ensure  lessons  learned  are  properly  vetted  and  passed  between  programs  to  alleviate 
schedule  growth  issues.  Furthermore,  future  analysis  similar  to  that  conducted  in  this 
study  can  target  specific  areas  for  schedule  improvement. 
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VI.  Future  Work 


Future  research  recommendations  include  an  in-depth  validation  of  the  entire 
ERAM  model.  While  the  space  launch  process  model  has  not  yet  been  fully  incorporated 
into  the  ERAM  model,  it  was  built  using  the  same  tool,  Extend  Sim,  with  the  intent  of 
incorporating  the  post-Milestone  C  portion  into  ERAM.  Issues  noted  in  ERAM  include 
an  inability  to  easily  model  a  negative  delay,  such  as  those  seen  due  to  priority  shifts  or 
accelerated  timelines  for  satellite  launches.  Additionally,  variable  naming  conventions 
used  in  ERAM,  or  lack-thereof,  create  difficulty  when  tracing  simulation  runs  through  the 
ERAM  model.  Lastly,  per  the  recommendation  of  Extend  Sim  technical  support,  the 
overall  ERAM  model  should  be  broken  down  into  more  manageable,  modular  pieces. 

In  the  future,  this  model  can  be  extended  to  encompass  other  launch  vehicles  such 
as  the  Delta  II  Medium  Launch  Vehicle  or  include  future  launch  capability  estimates  for 
Space  X,  Orbital  Sciences  and  other  potential  commercial  launch  vehicles.  The  inclusion 
of  other  launch  vehicles  may  yield  additional  insight  into  contributing  factors  for  delays. 
With  this  enhanced  knowledge,  specific  actions  can  be  taken  to  correct  satellite 
acquisition  schedules,  which  in  turn,  will  significantly  reduce  costs. 

There  exists  an  opportunity  for  significant  sensitivity  analysis  of  the  Space 
Launch  model  and  its  interaction  across  the  DoD  acquisition  process.  The  research  team 
recommends  future  research  focus  on  adjusting  the  many  variables  within  the  model  in 
order  to  minimize  overall  delay.  The  many  variables  include  distribution  parameters  and 
probabilities  of  delay  occurrence.  Ideally,  a  variable  may  be  found  of  which  its 
adjustment  will  have  a  disproportionate  effect  on  the  overall  delay.  In  addition,  future 
research  may  focus  on  adjusting  the  delay  categories  or  re-binning  the  individual  delays, 
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in  hopes  of  increasing  the  accuracy  of  the  model.  Additional  research  should  be 
completed  to  find  more  complete  documentation  of  program  timelines  and  associated 
delays.  Future  assessments  at  the  proper  classification  levels  should  be  accomplished  to 
enhance  the  opportunity  to  reduce  program  schedules. 
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Appendix  A:  Acronym  List 


AEHF 

Advanced  Extremely  High  Frequency 

AFI 

Air  Force  Instruction 

AFIT 

Air  Force  Institute  of  Technology 

AFSPC 

Air  Force  Space  Command 

CCAFS 

Cape  Canaveral  Air  Force  Station 

CDF 

Cumulative  Distribution  Function 

CLSRB 

Current  Launch  Schedule  Review  Board 

DoD 

Department  of  Defense 

DoF 

Degrees  of  Freedom 

DSCS 

Defense  Satellite  Communications  System 

EDF 

Empirical  Distribution  Function 

EELV 

Evolved  Expendable  Launch  Vehicle 

ERAM 

Enterprise  Requirements  and  Acquisition  Model 

FFRDC 

Federally  Funded  Research  and  Development  Corporation 

GAO 

Government  Accountability  Office 

GEO 

Geosynchronous 

GM 

Guidance  Memorandum 

GPS 

Global  Positioning  System 

GRP 

Group  Research  Project 

IC 

Intelligence  Community 

ILC 

Initial  Launch  Capability 

ISS 

International  Space  Station 

KS 

Kolmogorov-Smimov 

LCR 

Launch  Change  Request 

LCR 

Launch  Commit  Review 

LISN 

Launch  Information  Support  Network 

LV 

Launch  Vehicle 

MIT 

Massachusetts  Institute  of  Technology 

MUOS 

Mobile  User  Objective  System 

NLF 

National  Launch  Forecast 

NRO 

National  Reconnaissance  Office 

OTV-3 

Orbital  Test  Vehicle-3 

PDF 

Probability  Distribution  Function 

RCO 

Rapid  Capabilities  Office 

SBIRS 

Space  Based  InfraRed  System 

SLM 

Space  Launch  Manifest 

SMC 

Space  and  Missile  Systems  Center 

SME 

Subject  Matter  Expert 
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SOCOM 

Special  Operations  Command 

SPO 

System  Program  Office 

SV 

Satellite  Vehicle 

TRL 

Technology  Readiness  Level 

ULA 

United  Launch  Alliance 

VAFB 

Vandenberg  Air  Force  Base 

WX 

Weather 
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Appendix  B:  Sample  SME  Discussion 

SME  Name/Position/Organization: 

Interviewer: 

Date: 

Phone  #: 

1 .  Describe  your  roles  and  responsibilities  in  the  Space  Capability  Development 
process? 

2.  If  possible,  please  name  the  programs  you’ve  contributed  to? 

3.  How  early  and  where  in  your  program’s  timeline  was  the  launch 
capability/process  integrated? 

4.  Have  you  witnessed  a  program  delay  solely  due  to  the  launch  capability,  vehicle, 
process,  schedule,  weather,  etc?  If  so,  please  describe? 

5.  Does  your  organization  attempt  to  model  or  account  for  potential  program  delays 
due  specifically  to  space  launch? 

6.  In  your  experience,  where  does  a  launch  capability/process  have  the  highest 
likelihood  of  affecting  a  space  acquisition  program? 

7.  How  often  has  the  space  launch  timeline  been  affected  solely  by  your  program? 
Please  describe  if  able? 


8.  If  possible,  can  you  describe  the  duration  and  probability  of  launch  related  delays 
to  a  program? 

9.  Is  there  anything  else  you  can  add  with  respect  to  the  effects  of  the  launch 
capability/process  on  a  space  acquisition  program? 

10.  Can  you  recommend  any  specific  people  we  should  interview  with  respect  to 
program  delays  due  to  space  launch  (AFSPC,  SAF/AQ,  OSD  or  other  agencies)? 
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Appendix  C:  ERAM  2.4  Configuration  Changes 

Enterprise  Requirements  Acquisition  Model 
Configuration  Management  Worksheet 


This  form  provides  a  listing  of  the  development  and  the  changes  done  on  the  ERAM  Simulation  Model.  Use  the  table  below  to 
provide  the  simulation  software  used  (Arena  or  ExtendSim) ,  the  new  version  number,  the  name  of  the  author  and 
corresponding  organization,  the  date  of  revision  and  the  description  and  purpose  of  changes. 


Simulation 

Software 

Source 

Version 

Number 

New  Version 
Number 

Implemented 

By 

Org 

Date 

Description  of  Change 

Purpose  of 
Change 

-  Added  Space  Launch 
model  beneath  the  Rapid 
Acquisition  model. 


-  Changed  file  name  to 
ERAM  2.4  04162013.mox 

-  Changed  Excel  filename  to 
ERAM  2.4  04162013 

-  Added  Excel  file  “Space 
Launch  data”. 


Extensim 


2.30 


-  Modified  blocks  5823, 
5571,6071,6098,5539, 
5990,  5992  to  accommodate 
changes  in  Excel  filename. 


2.40 


Auger,  Baldus, 
Yoshimoto 


United  States 
Air  Force 
Institute  of 
Technology 


-  Added  blocks  6434,  6440 
to  enable  writing  into  Excel 
file  “Space  Launch  data”. 


04/16/13 


-  Added  tables  “Run  Space 
Launch”  and  “Space  Launch 
Delays”  to  ExtendSim 
database  to  enable  capturing 
of  Space  Launch  data. 


Accommodate 
Space  Launch 
Process  Delays 
add-ons 


-  Added  “Space  Launch” 
input  selector  to  Block  85, 
5773,  5996,  and  the  Excel 
Spreadsheet  (column  H). 


-  Added  “Buttons”  to 
ExtendSim  user  interface  for 
Space  Launch  relevant 
purposes. 


-  Modified  Block  358  to 
enable  tracking  of  “#  of 
runs”  from  the  general 
ERAM  model  and  exporting 
“Space  Launch  results”  to 
the  Excel  file  at  the  end  of 
the  simulation. 
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